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METHOD FOR PROVIDING PRIORITIZED 
DATA MOVEMENT BETWEEN ENDPOINTS 
CONNECTED BY MULTIPLE LOGICAL CHANNELS 

Technical Field 

The present invention relates to a data network, and more particularly, relates to a method 
for providing prioritized data movement between endpoints connected by multiple logical point- 
to-point channels in such a data network. 

Background 

A data network is generally consisted of a network of nodes connected by point-to-point 
links. Each physical link may support a number of logical point-to-point channels. Each channel 
may be a bi-directional communication path for allowing commands and data to flow between two 
connect nodes (e.g., hosts, I/O units and switch/switch elements) within the network. Each 
channel may refer to a single point-to-point connection where data may be transferred between 
endpoints (e.g., hosts and I/O units) in strict first-in, first-out (FIFO) order. Data may be 
transmitted in packets including groups called cells from source to destination often through 
intermediate nodes. In many data networks, cells between two endpoints (e.g., hosts and I/O 
units) may transverse the network along a given channel to ensure that cells are delivered in the 
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order in which they were transmitted. However, strict FIFO ordering of messages in such a data 
network causes a well known problem called "head-of-line blocking." Usually the "head-of-line 
blocking^problem arises when a high priority message is queued onto the tail of a FIFO queue, 
and has to wait for all other messages to be processed before the high priority message may reach 
5 the head of the FIFO queue for processing. As a result^ the overall performance of the data 
network can be significantly degraded. 

Therefore, there is a need for a more flexible, cost-effective, priority-driven and 
perfbrmance-efiBcient technique for providing prioritized data movement between endpoints 
ZZ^ connected by multiple logical channels in a data network, 

m SUMMARY 

I- Accordingly, various embodiments of the present invention are directed to a data network 

IZ and a method for providing prioritized data movement between endpoints connected by multiple 
logical channels in a data network. Such a data network may include a first node comprising a 
first plurality of first-in, first-out (FIFO) queues arranged for high priority to low priority data 
15 movement operations; and a second node operatively connected to the first node by multiple 
control and data channels, and comprising a second plurality of FIFO queues arranged in 
correspondence with the first plurality of FIFO queues for high priority to low priority data 
movement operations via the multiple control and data channels; wherein an I/O transaction is 
accompUshed by one or more control channels and data channels created between the first node 
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and the second node for moving commands and data for the I/O transaction during the data 
movement operations, in the order from high priority to low priority. 



BRIEF DESCRIPTION OF THE DRAWINGS 

A more complete appreciation of exemplary embodiments of the present invention, and 
many of the attendant advantages of the present invention, will become readily apparent as the 
same becomes better understood by reference to the following detailed description when 
considered in conjunction with the accompanying drawings in which like reference symbols 
indicate the same or similar components, wherein: 

FIG. 1 illustrates an example data network having several nodes interconnected by 
corresponding links of a basic switch; 

FIG, 2 illustrates another example data network having several nodes interconnected by 
corresponding links of a multi-stage switch; 

FIG. 3 illustrates an example data in groups of cells for communications according to an 
embodiment of the present invention; 

FIG, 4 illustrates an example data transfer between channel endpoints, for example, source 
node A and destination node B shown in FIGs. 1-2 connected by multiple logical point-to-point 
channels in strict first-in, first-out (FIFO) order; 

FIG. 5 illustrates an example implementation of data transfer between channel endpoints, 
source node A and destination node B connected by multiple logical point-to-point channels in 
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first-in, first-out (FIFO) order to provide prioritized processing of data movement operations 
according to an embodiment of the present invention; and 

FIG. 6 illustrates an example implementation of data transfer between channel endpoints, 
source node A and destination node B connected by multiple logical point-to-point channels in 
5 fu*st-in, first-out (FIFO) order to provide prioritized processing of data movement operations 
according to another embodiment of the present invention. 

C3 DETAILED DESCRIPTION 

The present invention is applicable for use v^ith all types of computer networks, I/O 
1 J channel adapters and chipsets, including follow-on chip designs which link together end stations 
CtO such as computers, servers, peripherals, storage devices, and communication devices for data 

communications. Examples of such computer networks may include a local area network (LAN), 
a v^de area network (WAN), a campus area network (CAN), a metropolitan area network 
;5 (MAN), a global area network (GAN) and a system area network (SAN), including newly 

developed computer networks using Next Generation I/O (NGIO) and Future I/O (FIO) and 
15 Server Net and those networks which may become available as computer technology advances in 
the future. LAN system may include Ethernet, FDDI (Fiber Distributed Data Interface) Token 
Ring LAN, Asynchronous Transfer Mode (ATM) LAN, Fiber Channel, and Wireless LAN. 
However, for the sake of simplicity, discussions will concentrate mainly on priority use of data 
movement in a simple data network having several example nodes (e.g., end stations including 
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computers, servers and I/O units) interconnected by corresponding links in compliance with the 
''Next Generation I/O Architecture'' for link specification and switch specification as set forth by 
the NGIO Forum on March 26, 1999, although the scope of the present invention is not limited 
thereto. 

5 Attention now is directed to the drawings and particularly to FIG. 1, a simple data 

network 10 having several interconnected nodes for data communications according to an 
embodiment of the present invention is illustrated. As shown in FIG. 1, the data network 10 may 
^ J include, for example, one or more centralized switches 100 and four different nodes A, B, C, and 
7l D, Each node (endpoint) may correspond to one or more I/O units and host systems including 
10 computers and/or servers. I/O unit may include one or more I/O controllers connected thereto. 
Co Each I/O controller may operate to control one or more I/O devices such as storage devices (e.g., 
H hard disk drive and tape drive). 

]/!_ The centralized switch 100 may contain switch ports 0, 1,2, and 3 each connected to a 

.'5 corresponding node of the four different nodes A, B, C, and D via a corresponding physical link 
15 1 10, 1 12, 1 14, and 116. Each physical link may support a number of logical point-to-point 

channels. Each channel may be a bi-directional communication path for allowing commands and 
data to flow between two connect nodes (e.g., host systems, I/O units and switch/switch 
elements) within the network. Each channel may refer to a single point-to-point connection 
where data may be transferred between endpoints (e.g., host systems and I/O units) in strict first- 
20 in, first-out (FIFO) order. The centralized switch 100 may also contain routing information using, 
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for example, explicit routing and/or destination address routing for routing data from a source 
node (data transmitter) to a destination node (data receiver) via corresponding link(s), and re- 
routing information for redundancy. 

The specific number and configuration of end stations (e.g., host systems and I/O units), 
5 switches and links shown in FIG. 1 is provided simply as an example data network. A wide 

variety of implementations and arrangements of an number of end stations (e.g., host systems and 
I/O units), switches and links in all types of data networks may be possible. 
S3 According to an example embodiment or implementation, the end stations (e.g., host 

1:1 systems and I/O units) of the example data network shown in FIG. 1 may be compatible with the 
10 "Next Generation Input/Output (NGIO) Specification'' as set forth by the NGIO Forum on March 
CO 26, 1999, According to the NGIO Specification, the switch 100 may be an NGIO fabric, and the 
[ end station may be a host system including one or more host channel adapters (HCAs) or an I/O 
IZ, unit including one or more target channel adapters (TCAs). 

'5 For example, FIG. 2 illustrates an example data network 10' using an NGIO architecture 

15 to transfer data from a source node to a destination node according to an embodiment of the 

present invention. As shown in FIG. 2, the data network 10' includes a multi-stage switch 100' 
comprised of a plurality of switches for allowing a host system and a target system to 
communicate to a large number of other host systems and target systems. In addition, any 
number of end stations, switches and links may be used for relaying data in groups of cells 
20 between the end stations and switches via corresponding NGIO links. 
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For example, node A may represent a host system 130. Similarly, node B may represent 
another network, including, but are not limited to, local area network (LAN), wide area network 
(WAN), Ethernet, ATM and fibre channel network, that is connected via high speed serial links. 
Node C may represent an I/O unit 170/ Likewise, node D may represent a remote system 190 
5 such as a computer or a server. Alternatively, nodes A, B, C, and D may also represent individual 
switches of the multi-stage switch 100' which serve as intermediate nodes between the host 
system 130 and the target systems 150, 170 and 190, 

^ ; The multi-state switch 100' may include a central network manager 250 connected to all 

the switches for managing all network management functions. However, the central network 

10 manager 250 may alternatively be incorporated as part of either the host system 130, the second 

£0 network 150, the I/O unit 170, or the remote system 190 for managing all network management 
functions. In either situation, the central network manager 250 may be configured for learning 

Jp, network topology, determining the switch table or forwarding database, detecting and managing 

J 1 faults or link failures in the network and performing other network management functions. 

15 A host channel adapter (HCA) 120 may be used to provide an interface between a 

memory controller (not shown) of the host system 130 and a multi-stage switch 100' via high 
speed serial NGIO links. Similarly, target channel adapters (TCA) 140 and 160 may be used to 
provide an interface between the multi-stage switch 100' and an I/O controller of either a second 
network 150 or an I/O unit 170 via high speed serial NGIO links. Separately, another host 

20 channel adapter (HCA) 180 may be used to provide an interface between a memory controller 
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(not shown) of the remote system 190 and the multi-stage switch 100' via high speed serial NGIO 
links. Both the host channel adapter (RCA) and the target channel adapter (TCA) may be 
implemented in compliance with "Afex/ Generation I/O Link Architecture Specification: HCA 
Specification, Revision L 0*' as set forth by NGIO Forum on July 20, 1999 for enabling the 
endpoints (nodes) to communicate to each other over an NGIO channel(s). However, NGIO is 
merely one example embodiment or implementation of the present invention, and the invention is 
not limited thereto. Rather, the present invention may be applicable to a wide variety of data 
networks, hosts and I/O units. 

The source node (data transmitter) may communicate with the destination node (data 
receiver) using a Virtual Interface Architecture (VI-A) in compliance with the ''Virtual Interface 
(VI) Architecture Specification, Version LO'' as set forth by Compaq Corp., Intel Corp., and 
Microsoft Corp,, on December 16, 1997. The VI Specification defines mechanisms for low- 
latency, high-bandwidth message-passing between interconnected nodes. Low latency and 
sustained high bandwidth may be achieved by avoiding intermediate copies of data and bypassing 
an operating system when sending and receiving messages. Other architectures may also be used 
to implement the present invention. 

FIG. 3 illustrates an embodiment of packet and cell formats of data transmitted fi-om a 
source node (data transmitter) to a destination node (data receiver) through switches and/or 
intermediate nodes according to the ''Next Generation I/O Link Architecture Specification/^ As 
shown in FIG. 3, a packet 300 may represent a sequence of one or more cells 3 10. Each cell 3 10 
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may include a fixed format header information 3 12, a variable format cell payload 3 14 and a cyclic 
redundancy check (CRC) information 316 The header information 312 may consist of 16 bytes of 
media control access information which specifies cell formation, format and validation. Each cell 
payload provides appropriate packet fields plus up to 256 bytes of data payload. The cell CRC 
5 may consist of 4-bytes of checksum for all of the data in the cell. Accordingly, the maximum size 
cell as defined by NGIO specification may be 292 bytes (256-byte Data Payload, 16-byte Header, 
16-Byte Virtual Address/Immediate data, and 4-byte CRC). 

FIG. 4 illustrates an example data transfer between channel endpoints, for example, source 
IZ node A (a particular host) and destination node B (an I/O unit) as shown in FIGs. 1-2 connected 
10 by multiple logical point-to-point channels in strict first-in, first-out (FIFO) order. These point- 
m to-point channels may be directly supported by the Virtual Interface Architecture (VI-A) and 
l'- NGIO. Many networking protocols (for example, Internet Protocol TCP/IP) provide for multiple 
^ I priorities of traffic to allow for varying types of information to pass between endpoints with 

varying precedence. Point-to-point connections as presented by the Virtual Interface Architecture 
15 (VIA) and the NGIO initiative provide only for FIFO ordering of messages. However, strict 
FIFO ordering as described, causes a "head-of-line blocking'' problem. This is because when a 
high priority message is queued onto the tail of a FIFO queue, such high priority message has to 
wait for all other messages to be processed before it reaches the head of the queue for processing. 
As a result, the overall performance of the data network can be significantly degraded. 
20 As shown in FIG. 4, node A may include, for example, physical FIFO queues (work 
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queues) 410 and 412 for either en-queuing or de-queuing data transfer requests and actual data 
transfer. Likewise, node B may include, for example, physical FIFO queues 420 and 422 for 
either en-queuing or de-queuing data transfer requests and actual data transfer. A logical I/O 
transaction between node A and node B may be accomplished by two channels 430 and 440, one 
channel for control and another channel for data. Each I/O transaction may consist of a I/O 
request for I/O services followed by data transfer (if indicated by the I/O service request) and a 
completion notification returned to the source node (initiator) of the I/O service request. The 
control channel 430 may support commands that describe data movement operations (i.e., sending 
I/O request and I/O reply messages). The data channel 440 actually moves the data between node 
A and node B, Since separate channels 430 and 440 are used for data transfer between channel 
endpoints, neither request nor reply messages need to wait for large blocks of data transmission 
between node A and node B. However, the "head-of-line blocking" and FIFO order will not 
allow prioritizing data once the data is queued on the control channel 430. 

Turning now to FIG. 5, the data transfer between channel endpoints, for example, source 
node A (a particular host) and destination node B (an I/O unit) connected by multiple logical 
point-to-point channels in first-in, first-out (FIFO) order to provide prioritized processing of data 
movement operations according to an embodiment of the present invention is illustrated. As 
shown in FIG. 5, node A may include, for example, physical FIFO queues (work queue in strict 
FIFO order) 5 lOA-5 ION in an order of priority (from high priority to low priority) and FIFO 
queue 512 for either en-queuing or de-queuing commands (data transfer requests) and actual data 
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transfer. Likewise, node B may include, for example, physical FIFO queues 520A-520N in an 
order of priority (from high priority to low priority) and FIFO queue 522 for either en-queuing or 
de-queuing commands (data transfer requests) and actual data transfer. A logical I/O transaction 
may be accomplished by a plurality of control channels 530A-530B created between node A and 
node B strictly for sending I/O request and I/O reply messages in the order from high priority to 
low priority, and a single data channel 540 created for moving data between node A and node B. 
Multiple control channels 530A-530B are used to prioritize command processing. Each control 
channel can be assigned a logical priority by the node (node A or node B) that is en-queuing the 
commands to be executed. For example, if assuming that only two priorities (high and low FIFO 
queues 5 lOA and 5 ION) are used, the node (node A or node B) that is en-queuing commands can 
use the low priority queue (for example^ FIFO queue 5 10 A) for normal traffic, and the high 
priority queue (for example, FIFO queue 5 ION) for urgent traffic. This allows high priority 
commands to move across the control channel while avoiding blocking behind low priority traffic. 

The specific number and configuration of FIFO queues and point-to-point channels 
between node A and node B shown in FIG. 5 is provided simply as an example priority level of 
data movement between endpoints in an example data network. A wide variety of 
implementations and arrangements of any number of data channels and control channels between 
endpoints in all types of data networks may be possible. For example, the priority model shown in 
FIG. 5 can also be extended to allow for multiple data channels, each assigned a different priority 
level. This allows for prioritized data to be mapped onto prioritized data channels, and for data of 
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differing priorities to move independently across different data channels between endpoints in an 
example data network. 

FIG. 6 illustrates the data transfer between channel endpoints, for example, source node A 
(a particular host) and destination node B (an I/O unit) connected by multiple logical point-to- 
point channels in first-in, first-out (FIFO) order to provide prioritized processing of data 
movement operations according to another embodiment of the present invention. As shown in 
FIG. 6, node A may include a FIFO queue (work queue in strict FIFO order) 610 and FIFO 
queues 612A-612N in an order of priority (from high priority to low priority) for either en- 
queuing or de-queuing commands (data transfer requests) and actual data transfer. Likewise, 
node B may include a FIFO queue 620 and FIFO queues 622A-622N in an order of priority for 
either en-queuing or de-queuing commands (data transfer requests) and actual data transfer. 

A logical I/O transaction may be accomplished by a single control channel 630 created 
between node A and node B strictly for sending I/O request and I/O reply messages, and a 
plurality of data channels 640A-640N created for moving data between node A and node B in the 
order from high priority to low priority. A single control channel 630 may be sufficient and 
desirable, but data transfer spread between multiple data channels 640A-640N can significantly 
decrease latency and increase bandwidth. Moreover, dividing data transfer between different data 
channels may help overall I/O responsiveness and distribute even loading in the data network. 
Multiple data channels 640A-640N are used to prioritize data processing. Each data channel can 
be assigned a logical priority by the node (node A or node B) that is en-queuing the data to be 
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transferred. The number of data channels used for data movement between node A and node B 
may be assigned by any given node when the channels are created. 

If node A and node B are channel endpoints (e.g., host systems and I/O units) of an 
example data network shown in FIG. 2 implemented in compliance with the "Next Generation 
Input/Output (NGIO) Specification'', each cell may contain a 3 -bit priority indication as part of 
the 16 byte Media Access Control (MAC) header shown in FIG. 3 for providing, for example, a 
maximum eight (8) levels of priority. However, currently only five of the eight possible 
combinations are defined by NGIO protocol. The highest level priority may be reserved for 
management packets. The lowest level priority may be Priority "0** for best effort. Next to the 
lowest priority may be Priority "1" for privileged best effort, Priority "2" for negotiated normal 
latency, and Priority "3" for negotiated minimum latency. Management class of service may be 
provided to allow system administrators to communicate with all nodes connected to the NGIO 
fabric. These priorities may be absolute, meaning that a higher priority will always preempt a 
lower priority. 

For example, source node A may transmit all data fi'om FIFO queues configured to 
transmit at management service before any data is sent from FIFO queues configured to transmit 
at best effort or privileged best effort service. Each FIFO queue shown in FIGs. 5 and 6 may be 
assigned to one of the five priorities based on the end-to-end class-of-service and/or the quality- 
of-service desired for that FIFO queue. Each node (node A or node B) may include one or more 
channel adapters configured with a multiplexing function based on priority for multiplexing and 
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transmitting back to back cells of the same priority from multiple FIFO queues through the 
assigned control or data channels. 

As described from the ft^regoing, the present invention advantageously provides a unique 
cost-effective and performance-efficient solution for prioritized data movement between 
endpoints connected by multiple logical channels in a data network. Such a prioritized data 
movement solution is especially important for connections between a host computer and a node 
that provides inter-networking to external networks running industry standard protocols such as 
TCP/IP. Moreover, such a prioritized data movement solution is also critical for implementation 
of networking products that allow for end-to-end class-of-service and/or quality-of- service 
between an NGIO based host computer and another computer on a LAN or WAN. 

While there have been illustrated and described what are considered to be exemplary 
embodiments of the present invention, it will be understood by those skilled in the art and as 
technology develops that various changes and modifications may be made, and equivalents may be 
substituted for elements thereof without departing from the true scope of the present invention. 
For example, the present invention is applicable to all types of redundant type networks, 
including, but is not limited to, Next Generation Input/Output (NGIO), ATM, SAN (system area 
network, or storage area network), server net, Future Input/Output (FIO), fiber channel, and 
Ethernet, Many modifications may be made to adapt the teachings of the present invention to a 
particular situation without departing from the scope thereof Therefore, it is intended that the 
present invention not be limited to the various exemplary embodiments disclosed, but that the 
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present invention includes all embodiments falling within the scope of the appended claims. 
What is claimed is: 
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CLAIMS 

1 . A network, comprising; 

a first node comprising a first plurality of first-in, first-out (FIFO) queues arranged for 
high priority to low priority data movement operations; and 

a second node operatively connected to said first node by multiple control and data 
channels, said second node comprising a second plurality of FIFO queues arranged in 
correspondence with said first plurality of FIFO queues for high priority to low priority data 
movement operations via said multiple control and data channels; 

wherein an I/O transaction is accomplished by one or more control channels and data 
channels created for moving commands and data separately between said first node and said 
second node during said data movement operations, in the order fi*om high priority to low 
priority. 

2. A network as claimed in claim 1, wherein said control channels are used to 
prioritize command processing, and each control channel is assigned with a different priority by 
one of the nodes that is en-queuing the commands to be executed based on the quality of service 
desired. 

3. A network as claimed in claim 2, wherein said first plurality of FIFO queues 
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include a high priority FIFO queue provided to queue commands for urgent traffic, a low priority 
FIFO queue provided to queue commands for normal traffic, and another FIFO queue provided to 
queue data for data movement operations. 

4. A network as claimed in claim 1, wherein said data is transmitted in groups of cells 
with each cell having a header utilized for indicating whether the cell is transmitted in a priority 
order, 

5. A network as claimed in claim 4, further comprising a multi-stage switch which 
comprises a plurality of different routes for connecting said first node and said second node for 
data movement operations. 

6. A network as claimed in claim 4, wherein each of said first node and said second 
node includes one or more channel adapters configured with a multiplexing function based on a 
priority order for multiplexing and transmitting back to back cells of the same priority from 
multiple FIFO queues through assigned control or data channels. 

7. A network as claimed in claim 2, wherein said data is spread between multiple data 
channels to prioritize data processing while concomitantly decreasing latency and increasing 
bandwidth during said data movement operations. 
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8. A network as claimed in claim I, wherein said first node and said second node are 
channel endpoints of the network implemented in compliance with the ''Next Generation 
Input/Output (NGIO) Specification'\ 

9. A network as claimed in claim 8, wherein said multiple control and data channels 
are directly supported by the ''Virtual Interface (VI) Architecture Specification^^ and the "Next 
Generation Input/Output (NGIO) Specification'' for multiple priorities of traffic to allow for 
varying types of information to pass between endpoints with varying precedence. 

10. A network arrangement, comprising: 
a host system; 

at least one remote system; 

a switch fabric comprising a plurality of different switches which interconnect said host 
system to said remote system along different control and data channels for data movement 
operations; 

said host system comprising multiple first-in, first-out (FIFO) queues arranged for high 
priority to low priority data movement operations; and 

said remote system comprising multiple first-in, first-out (FIFO) queues arranged for high 
priority to low priority data movement operations; 
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wherein an I/O transaction is accomplished by one or more control channels and data 
channels created for moving commands and data separately between said host system and said 
remote system during said data movement operations, in the order from high priority to low 
priority. 

11. A network arrangement as claimed in claim 10, wherein said control channels are 
used to prioritize command processing, and each control channel is assigned with a different 
priority by one of the nodes that is en-queuing the commands to be executed based on the quality 
of service desired. 

12. A network arrangement as claimed in claim 10, wherein said FIFO queues of one 
of said host system and said remote system include a high priority FIFO queue provided to queue 
commands for urgent traflSc, a low priority FIFO queue provided to queue commands for normal 
traffic, and another FIFO queue provided to queue data for data movement operations. 

13. A network arrangement as claimed in claim 10, wherein said data is transmitted 
from said host system to said remote system in groups of cells with each cell having a header 
utilized for indicating whether the cell is transmitted in a priority order. 

14. A network arrangement as claimed in claim 13, wherein each of said host system 
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and said remote system includes one or more channel adapters configured with a multiplexing 
fiinction based on a priority order for multiplexing and transmitting back to back cells of the same 
priority from multiple FIFO queues through assigned control or data channels. 

15. A network arrangement as claimed in claim 10, wherein said data is spread 
between multiple data channels to prioritize data processing while concomitantly decreasing 
latency and increasing bandwidth during said data movement operations. 

16. A network arrangement as claimed in claim 10, wherein said host system and said 
remote system are channel endpoints of the network implemented in compHance with the ''Next 
Generation Input/Output (NGIO) Specification'^ 

17. A network arrangement as claimed in claim 10, wherein said multiple control and 
data channels are directly supported by the ''Virtual Interface (VI) Architecture Specification'" 
and the "Next Generation Input/Output (NGIO) Specification'' for multiple priorities of traflSc to 
allow for varying types of information to pass between said host system and said remote system 
with varying precedence. 

18. A method for providing prioritized data movement between a source node and a 
destination node in a data network, said method comprising: 
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establishing one or more control channels and one or more data channels between said 
source node and said destination node for transferring commands that describe data movement 
operations from said source node to said destination node and for moving data from said source 
node to said destination node; 

assigning a logical priority to each control channel for transferring high priority commands 
to move across the control channel before low priority commands during said data movement 
operations; and 

transferring, at said source node, data in groups of cells to said destination node along the 
data channel. 

19. A method as claimed in claim 18, further comprising; 

assigning a logical priority to each data channel for high priority data to move across the 
data channel before low priority data during said data movement operations; and 

moving, at said source node, high priority data in groups of cells to said destination node 
along the data channel before low priority data during said data movement operations. 

20. A method as claimed in claim 1 8, wherein said data is transmitted from said source 
node to said destination in groups of cells with each cell having a header utilized for indicating 
whether the cell is transmitted in a priority order. 
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21. A node comprising: 

a first plurality of first-in, first-out (FIFO) queues arranged for high priority to low priority 
data movement operations; and 

an interface for operatively coupling said node to another node by multiple control and 
data channels^ such that an 1/0 transaction is accomplished, by one or more control channels and 
data channels created for moving commands and data separately between said node and said 
another node during said data movement operations, in order fi-om high priority to low priority. 

22. A computer-readable medium that stores computer-executable instructions for 
transferring commands and data fi-om a first node to a second node, said instructions causing a 
computer to: 

arrange a first plurality of first-in, first-out (FIFO) queues, located within the first node, 
for high priority to low priority data movement operations; and 

operatively couple said first node to said second node by multiple control and data 
channels, such that an I/O transaction is accomplished, by one or more control channels and data 
channels created for moving commands and data separately between said node and said another 
node during said data movement operations, in order fi-om high priority to low priority, 

23 . A computer-readable medium that stores computer-executable instructions for 
transferring commands and data from a source node to a destination node, said instructions 
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causing a computer to: 

establish one or more control channels and one or more data channels between said source 
node and said destination node for transferring commands that describe data movement 
operations from said source node to said destination node and for moving data from said source 
node to said destination node; 

assign a logical priority to each control channel for transferring high priority conmiands to 
move across the control channel before low priority commands during said data movement 
operations; and 

transfer, at said source node, data in groups of cells to said destination node along the data 
channel. 
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ABSTRACT OF DISCLOSURE 

A data network and a method for providing prioritized data movement between endpoints 
connected by multiple logical channels. Such a data network may include a first node comprising 
a first plurality of first-in, first-out (FIFO) queues arranged for high priority to low priority data 
movement operations; and a second node operatively connected to the first node by multiple 
control and data channels, and comprising a second plurality of FIFO queues arranged in 
correspondence with the first plurality of FIFO queues for high priority to low priority data 
movement operations via the multiple control and data channels; wherein an I/O transaction is 
accomplished by one or more control channels and data channels created between the first node 
and the second node for moving commands and data for the I/O transaction during the data 
movement operations, in the order from high priority to low priority. 
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DECLARATION AND POWER OF ATTORNEY FOR PATENT 

APPLICATION 
(FOR INTEL CORPORATION PATENT APPLICATIONS) 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below, next to 
my name, 

I believe I am the original, first, and sole inventor (if only one name is listed 
below) or an original, first, and joint inventor (if plural names are listed 
below) of the subject matter which is claimed and for which a patent is 
sought on the invention entitled 

METHOD FOR PROVIDING PRIORITIZED DATA MOVEMENT 
BETWEEN ENDPOINTS BY MULTIPLE LOGICAL CHANNELS 

the specification of which 

X is attached hereto. 

was filed on as 

United States Application Number 

or PCT International Application Number and was 

amended on . 

(if appUcable) 

I hereby state that I have reviewed and understand the contents of the above- 
identified specification, including the claim(s), as amended by any amendment 
referred to above. I do not know and do not believe that the claimed 
invention was ever known or used in the United States of America before my 
invention thereof, or patented or described in any printed pubUcation in any 
country before my invention thereof or more than one year prior to this 
application, that the same was not in public use or on sale in the United 
States of America more than one year prior to this application, and that the 
invention has not been patented or made the subject of an inventor's 
certificate issued before the date of this application in any country foreign to 
the United States of America on an application filed by me or my legal 
representatives or assigns more than twelve months (for a utility patent 
appUcation) or six months (for a design patent application) prior to this 
application. 

I acknowledge the duty to disclose all information known to me to be 
material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1.56. 
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I hereby claim foreign priority benefits under Title 35, United States Code, 
Section 119(a)-(d), of any foreign application(s) for patent or inventor's 
certificate listed below and have also identified below any foreign application 
for patent or inventor's certificate having a filing date before that of the 
application on which priority is claimed; 

Priority 

Prior Foreign Application(s) Claimed 



(Number) (Country) (Day/MonthA'ear Filed) Yes No 



I hereby claim the benefit under title 35, United States Code, Section 1 19(e) 
of any United States provisional application(s) listed below 



(Application Number) Filing Date 

I hereby claim the benefit under Title 35, United States Code, Section 120 of 
any United States appHcation(s) listed below and, insofar as the subject 
matter of each of the claims of this application is not disclosed in the prior 
United States application in the manner provided by the first paragraph of 
Title 35, United States Code, Section 112, I acknowledge the duty to 
disclose all information known to me to be material to patentability as defined 
in Title 37, Code of Federal Regulations, Section 1.56 which became 
available between the filing date of the prior application and the national or 
PCT international filing date of this application: 



(Application Number) Filing Date (Status - patented , 

pending, abandoned) 
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DEC-15-1999 11:00 INTEL SnL 

I hereby appoint I>onaldR, Antt>3ieili,Rcg,No.:iUv*tyw,i^a»iu ^ 6?"^^ 6700 P. 02/03 

Reg, No, 22,4$6; Willkm 1. Solomon, Reg. No. 28,565; Gregory E. Montone, Reg. No, 28,141; Rojtald J. 

Shore, Reg. No. 28,577; l>3nald E Stout, Rcg- No. 26,422; Alan E. ScMavelU, Reg. No. 32»0S7; James N. 

Dresser, Reg. No. 22,973; Carl L Brundidgc, Rrrg, No, 29,621; Paul J, Skwierawski, Reg, No, 32J73. my 
. atttaneys-. of ANTONELU TERRY, STOUT & KRAUS, LLP with offices located at 1300 Nailii 

Seventeenth Sti^et, Suite iSOO, Arlington, Virginia 22209, telephone: (703) 312-6600, fax: (703) 312-6666; 

and Alan Aldous, Reg. No. 31,905; Robert D. Anderson, Reg. No. 33,S26; Joseph R. Bond, Reg. No. 

36,458; Richard C. Calderwood, Reg. No. 35,468; Jeffrey S. Dracger, Reg. No, 41,000; Cynthia Thomas 

Faatz, Re^ No. 39.973; Sean Fit^erald, Reg. No. 32,027; Seth Z. Kalson, Reg. Na 40,670; David J. 

K^lan, Reg. No. 4i,l05; Leo V. Novakoski, Reg. No. 37,198; Naomi Obinata, J^g, No. 39,320; Thomas C. 

Reynolds, Reg. No. 32,488; Steven P. Skabmt Reg. No. 36 Jt79; Howard A. Sfcaist* Reg. Na 36,008; Steven 

C. Stewart, Reg, No. 33,555; Raymond I Wemcr, Reg. No. 34,752; and Charles K. Young, Reg. No. 

39,435; my patent attorneys, md Calvin E. Weils, Reg, No. P43i56; and Alexander Ulysses Witkowski. 

Reg. No. P43^80; my patent agents, of INTEL CORPORATION; with fuU power of substitution and 

revocation, to prosecute this application and to transact all business in the Patent and Trademark OfHce 

connected herewith. 

Send ail carrespondence to; 

ANTONELLI, TBRRY, STOUT Sc KRAUS, LLP 
1300 North Seventeenth Street 
Suite 1800 
Arlington, VA. 22209 

Direct all telephone calls and faxes to: 

TEL: (703)312-6600 
FAX: (703) 312-6666 

I hereby declare that all statements nsade herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and further that these statements were made with the 
knowledge that willful false statements and the like so made are punishable by fine or imprisonment, or 
both, under Section 1001 of Title 18 of the United States Code and that such wiUfiil false statements may 
jeopardize ^ validity of the application or any patent issued thereon. 
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{City, State) 

Post Office Addres s 16965 NW Bemietta Ct, Portland, Oregon 97229 



Citizenshi p United States 



itedSfetes 



(Coimtiy) 



Full Name of Second/Joint Invento r Jefl&cv M. BUTLER 
Inventor's Signature _ 



Residence Seaverton, Oregon 



(City, State) 

Post Office Addres s 2402 NW Schmidt Way. #190. Beaverton, OR 97006 



Date 

Citizcnslu p United States 

(Country) 



Full Name of Third/Joint Invento r Dave B. MINTURN 
Inventor's Sifinature '^T^ M^^!eS 



Residenc e Hillsboro^ Oregon 



Date Xa/y^/i^^^ 



(City,S^e) 

Post Office Addres s 17125 Chegalem Wav, HillsbofQ. Qreg(m 97123 



Citizenshii ) United States 
Conntry) 



FuU Name of Fomth/Joint Inventor^ 

Inventor's Signature 

Residence 



Date 



(City, Slate) 
Post Office Address 



(Country) 



^ Citizenship, 
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DEC-15-1999 1?:14 
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I hereby appoint: Etonald R. AntoneUi, Reg. No. 20,296; David T, Terry, Reg. No. 20.178; Melvin Kraus, 
Reg. No. 22,466; William I. Solomon, Reg, No. 28,565; Gregory E. Montoae, Reg. No. 28,141; Ronald J. 
Shore, Reg. No. 28.577; Donald E, Stout, Reg, No. 26,422; Alan E. Schiavelli, Reg, No. 32,087; James N. 
Dresser, R^. No. 22,973; Carl I. Bnmdidge, Reg. No. 29,621; Paul J. Skwierawski, Reg. No. 32,173, my 
attorneys; of ANTONELLI, TERRY, STOUT & KRAUS, LLP with offices located at 1300 North 
Seventeenth Street, Suite 1800, Arlington, Virginia 22209, telephone: (703) 312-6600, fax: (703) 312-6666; 
and Alan K, Aldoiis, Reg. No. 31,905; Robert D. Anderson, Reg. No. 33,826; Joseph R. Bond, Reg. No. 
36,458; Richard C. Calderwood, Reg. No, 35,468; Jeffrey S. Draeger, Reg. No. 41,000; Cynthia Thomas 
Faatz, Reg No, 39,973; Sean Fitzgerald, Reg. No, 32,027; Seth Z. Kalson, Reg. No. 40,670; David J. 
Kaplan, Reg. No. 41,105; Leo V. Novakoski, Reg. No. 37,198; Naomi Obmata, Reg. No. 39,320; Thomas C, 
Reynolds, Reg. No. 32,488; Steven P. Skabrat, Reg. No. 36,279; Howard A. Skaist, Reg. No. 36,008; Steven 
C, Stewart, Reg. No, 33,555; Raymond J. Werner, Reg. No, 34,752; and Charles K, Young, Reg. No. 
39,435; my patent attorneys^ and Calvin E. Wells, Reg. No. P43,256; and Alexander Ulysses Witkowski, 
Reg. No. P43,280; my patent agents, of INTEL CORPORATION; with full power of substitution and 
revocation, to prosecute this application and to transact all business in the Patent and Trademark Office 
connected herewith. 

Send ail correspondence to: 

ANTONELLI, TERRY, STOUT & KRAUS, LLP 
1300 Nor& Seventeenth Street 
Suite 1800 
Arlington, VA. 22209 

Direct all telephone calls and faxes to: 

TEL: (703)312-6600 
FAX: (703) 312-6666 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made 
on information and belief are believed to be true; and forther that these statements were made with the 
knowledge that willful felse statements and the like so made are punishable by fine or imprisonment, or 
both, under Section 1001 of Title 18 of the United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued thereon. 
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Inventor's Signature Date 

Residence Portland. Qr^on Citizenshi p United States 

(City, State) (Country) 
Post Office Addres s 16965 NW Bemietta Ct. Portland. Oregon 97229 
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Inventor's J 

Residence Beaverton. Qi@oo^ ' Citizenshi p Oiuted States 

(City, State) (Country) 
Post Office Address 2402 NW Schmidt Way. #190. Beav^ton, OR 97006 



Full Name of Third/Joint hiventor Pave B . MINTURN 



Inventor's Signature Date 

Residenc e Hillsboro. Oregon Citizenshi p United States 

(City, State) Country) 
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Title 37, Code of Federal Regulations, Section 1.56 
Duty to Disclose Information Material to Patentability 



(a) A patent by its very nature is affected with a public interest. The public interest is best served, 
and the most effective patent examination occxirs when, at the time an application is being examined, the Office 
is aware of and evaluates the teachings of all information material to patentability. Each individual associated 
with the filing and prosecution of a patent application has a duty of candor and good faith in dealing with the 
Office, which includes a duty to disclose to the Office all information known to that individual to be material to 
patentability as defined in this section. The duty to disclosure information exists with respect to each pending 
claim until the claim is cancelled or withdrawn from consideration, or the application becomes abandoned. 
Information material to the patentability of a claim that is cancelled or withdrawn torn consideration need not be 
submitted if the information is not material to the patentability of any claim remaining under consideration in the 
application. There is no duty to submit information which is not material to the patentability of any existing 
claim. The duty to disclosure all information known to be material to patentability is deemed to be satisfied if 
all information known to be material to patentability of any claim issued in a patent was cited by the Office or 
submitted to the Office in the manner prescribed by cfal.97(bHd) and 1.98. However, no patent will be granted 
on an application in connection with which fraud on the Office was practiced or attempted or the duty of 
disclosure was violated through bad faith or intentional misconduct. The Office encourages applicants to 
carefully examine: 

( 1 ) Prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) The closest information over which individuals associated with the filing or prosecution of a 
patent application believe any pending claim patentably defines, to make sure that any material information 
contained therein is disclosed to the Office. 

(b) Under this section, information is material to patentability when it is not cumulative to 
information already of record or being made or record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of 
impatentability of a claim; or 

(2) It refutes, or is inconsistent with, a position the applicant takes in: 

(i) Opposing an argument of impatentability relied on by the Office, or 

(ii) Asserting an argument of patentability. 

A prima facie case of unpatentability is established when the information compels a conclusion that a claim is 
unpatentable under the preponderance of evidence, burden-of-proof standard, giving each term in the claim its 
broadest reasonable construction consistent with the specification, and before any consideration is given to 
evidence which may be submitted in an attempt to establish a contrary conclusion of patentability. 

(c) Individuals associated with the filing or prosecution of a patent application within the 
meaning of this section are; 

(1 ) Each inventor named in the application; 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the 
application and who is associated with the inventor, with the assignee or with anyone to whom there is an 
obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by 
disclosing information to the attorney, agent, or inventor. 



